Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow customization and extension of lockfile metadata headers #106

Closed
wants to merge 4 commits into from

Conversation

maresb
Copy link
Contributor

@maresb maresb commented Sep 5, 2021

Here is an explanation of this PR through examples, showing only the resulting headers.

I have not yet written any tests or updated the README. Do these features look good?

conda-lock -f environment.yml --metadata=previous
# Generated by conda-lock.
# platform: win-64
# input_hash: f15a045753a401da73dd7c1693fd031e0ad41c0b4c9ca8545c0a8ab56c21d16c
@EXPLICIT

conda-lock -f environment.yml
# ---
# conda-lock-metadata:
#   about: This lockfile was generated by conda-lock to ensure reproducibility.
#   platform: win-64
#   input_hash: f15a045753a401da73dd7c1693fd031e0ad41c0b4c9ca8545c0a8ab56c21d16c
# ...
@EXPLICIT

conda-lock -f environment.yml "--comment=$(cat README.txt)"
# ---
# conda-lock-metadata:
#   about: This lockfile was generated by conda-lock to ensure reproducibility.
#   platform: win-64
#   input_hash: f15a045753a401da73dd7c1693fd031e0ad41c0b4c9ca8545c0a8ab56c21d16c
#   comment: |-
#     Contents of README.txt
#     ¡the second line! 😃
# ...
@EXPLICIT

conda-lock -f environment.yml --metadata=all
# ---
# conda-lock-metadata:
#   about: This lockfile was generated by conda-lock to ensure reproducibility.
#   command: conda-lock -f environment.yml --metadata=all
#   command_with_path: /root/conda/envs/conda-lock/bin/conda-lock -f environment.yml --metadata=all
#   comment: null
#   conda_lock_version: 0.11.3.dev0+gf2ba8d4.d20210904
#   created_by: root
#   input_hash: f15a045753a401da73dd7c1693fd031e0ad41c0b4c9ca8545c0a8ab56c21d16c
#   platform: win-64
#   timestamp: 2021-09-05 23:43:18+02:00
# ...
@EXPLICIT

conda-lock -f environment.yml --metadata=created_by,comment,command,timestamp --comment="Run the following command to update this project's dependencies."
# ---
# conda-lock-metadata:
#   created_by: mares
#   comment: |-
#     Run the following command to update this project's dependencies.
#   command: conda-lock -f environment.yml --metadata=created_by,comment,command,timestamp '--comment=Run the following command to update this project'"'"'s dependencies:'
#   timestamp: 2021-09-05 23:51:07+02:00
# ...
@EXPLICIT

@mariusvniekerk
Copy link
Collaborator

This would almost certainly have to be an opt-in not an opt-out feature

@maresb
Copy link
Contributor Author

maresb commented Sep 8, 2021

@mariusvniekerk, easy enough if I set the default to "previous". Any other feedback?

@mariusvniekerk
Copy link
Collaborator

mariusvniekerk commented Sep 8, 2021

We might want to explicitly version the metadata header version (with no version being the simple legacy form). That leaves an extension point for the future open.

@maresb
Copy link
Contributor Author

maresb commented Sep 8, 2021

I implemented your suggestions.

I realize now that I may have misinterpreted your suggestion for versioning metadata...

Do you mean like there would be a flag --metadata-version=n and in the future we support all previous versions of metadata? If so, that seems like a good idea. Maybe it would make sense to have --metadata-version default to latest so that pinning is opt-in, but in the absence of --metadata it defaults to v0.

What sort of versioning would make sense? I'm thinking semver major-only so that we'd only increment when there's a breaking change.

The new behavior of my two respective commits:

conda-lock -f environment.yml
# Generated by conda-lock.
# platform: win-64
# input_hash: f15a045753a401da73dd7c1693fd031e0ad41c0b4c9ca8545c0a8ab56c21d16c
@EXPLICIT

conda-lock -f environment.yml --metadata=all
# ---
# conda-lock-metadata:
#   about: This lockfile was generated by conda-lock to ensure reproducibility.
#   command: conda-lock -f environment.yml --metadata=all
#   command_with_path: /root/conda/envs/conda-lock/bin/conda-lock -f environment.yml --metadata=all
#   comment: null
#   conda_lock_version: 0.11.3.dev0+gf2ba8d4.d20210904
#   created_by: root
#   input_hash: f15a045753a401da73dd7c1693fd031e0ad41c0b4c9ca8545c0a8ab56c21d16c
#   metadata_version: 1
#   platform: win-64
#   timestamp: 2021-09-08 20:12:23+02:00
# ...
@EXPLICIT

@mariusvniekerk
Copy link
Collaborator

mariusvniekerk commented Sep 8, 2021

Yeah that looks good to me as an output. Maybe not emit the null portions for the metadata, by default.

There are some parts of conda-lock that do parse out fields like the input hash and I suspect those tests may be need some updating

@maresb
Copy link
Contributor Author

maresb commented Sep 8, 2021

By my logic, I think comment should be included even when null. My reason is that all is an alias for all fields, and that way you can use it to view all the available fields as a sort of documentation. Afterwards, if you care about privacy and data leakage, you should select just the ones you want to keep. But if you're not convinced I'll change it.

Regarding the hash filtering in the tests, the output should now be identical unless I made a mistake.

Unfortunately I won't have any time until next week to work on this.

@wolfv
Copy link

wolfv commented Oct 12, 2021

@maresb maybe you're interested in chiminig in here: mamba-org/mamba#1209

@mariusvniekerk
Copy link
Collaborator

@maresb Now that the new lock stuff is mostly live, this can probably be resurrected if you still want it

@maresb
Copy link
Contributor Author

maresb commented Mar 9, 2022

Cool, thanks! Unfortunately it will be a while before I can get to this.

@maresb
Copy link
Contributor Author

maresb commented Sep 24, 2022

This is stale.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants